Capturing spatial interdependence in image features: the counting grid, an epitomic representation for bags of features

摘要

In recent scene recognition research images or large image regions are oftenrepresented as disorganized "bags" of features which can then be analyzed usingmodels originally developed to capture co-variation of word counts in text.However, image feature counts are likely to be constrained in different waysthan word counts in text. For example, as a camera pans upwards from a buildingentrance over its first few floors and then further up into the sky Fig. 1,some feature counts in the image drop while others rise -- only to drop againgiving way to features found more often at higher elevations. The space of allpossible feature count combinations is constrained both by the properties ofthe larger scene and the size and the location of the window into it. Tocapture such variation, in this paper we propose the use of the counting gridmodel. This generative model is based on a grid of feature counts, considerablylarger than any of the modeled images, and considerably smaller than the realestate needed to tile the images next to each other tightly. Each modeled imageis assumed to have a representative window in the grid in which the featurecounts mimic the feature distribution in the image. We provide a learningprocedure that jointly maps all images in the training set to the counting gridand estimates the appropriate local counts in it. Experimentally, wedemonstrate that the resulting representation captures the space of featurecount combinations more accurately than the traditional models, not only whenthe input images come from a panning camera, but even when modeling images ofdifferent scenes from the same category.